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ABSTRACT 

Development and evaluation of interactive videodisc 
(IVD) courseware for Engl i sh-as*“a*“Second*“Language (ESL) instruction 
are described. Effective listening and viewing strategies, identified 
through descriptive research, were incorporated into an existing ESL 
IVD. Subjects, 81 college ESL students, were pre-tested, then divided 
into control and experimental groups. The control group worked on the 
original IVD version, the experimental group on the revised version. 
After online practice, all subjects took a posttest. Then 33 of the 
81 were further divided into two groups, one receiving another 
session of individual online strategy training as before, and the 
other working in a small group on the revised version. A delayed 
posttest focusing on comprehension and recall was then admini stered. 
No differences were found in the effects of strategy training or 
group/collaborative learning. Questions in the pretest, posttest, and 
delayed posttest elicited responses about strategy use; these were 
analyzed for insight into strategy training. It is concluded that the 
teaching of learning strategies is sensitive to many variables. A 
major limitation seen in this study is lack of training time for 
subjects. However, it is concluded that this approach to 
inco) porat ing computer-assisted language learning strategies into the 
regular curriculum is useful. Contains 38 references. (MSE) 
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Evaluation of interactive videodisc courseware: 
A strategy perspective 
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ABSTRACT 

The paper addresses the evaluation issues of a piece of interactive 
videodisc courseware for computer-assisted language learning 
purposes from the perspective of learning strategy research. The 
study investigates strategy use by examining comparison 
experimental data. In a study of the research project (NSC 83- 
0301-H007-016) in its first phase, effective listening and viewing 
strategies were identified and classified through descriptive 
research techniques, and then implemented into a revised version 
of the courseware. Then, 81 EFL subjects classified into a 
control group and an experiment group were recruited to 
participate in the research. After they took the pretest, the control 
group were asked to use the non-strategy version of the 
courseware; the experiment group, the strategy version. After 
on-line practice, subjects took a posttest*. In the second part of 
the comparison design, 33 subjects out of the 81 were further 
divided into two groups. The first group received another session 
of individual on-line non-strategy training as before, and the 
second group who worked with peers collaboratively received the 
strategy version. Last, a delayed posttest was administered". All 
the measurements of subject performance in the paper included 
comprehension questions and a recall summary. No differences 
were found as to effects of strategy training or collaborative 
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' The results of this part were reported in TESOL ’95, Long Beach, CA, USA (Liou, 
1995a) during March 28 to April. No proceedings was published for the conference. 



^ The results of this part were reported in CALICO ’95, Middlebury College, VT, USA 
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learning. Three questionnaires were used in the pretest, posttest, 
and delayed posttest sessions to elicit responses about strategy 
use. The results shed some light on the effects of strategies 
instruction and collaborative learning. Future directions and 
implications are discussed. 

I. INTRODUCTION 

The paper addresses evaluation issues of interactive videodisc (IVD) for 
computer-assisted language learning (CALL) purposes. Recently, learning 
process becomes of paramount importance and learning strategies, among other 
factors, are believed to shed light on the process. Yet, little of strategy 
research in CALL environments has been conducted, and further, video viewing 
comprehension process has seldom been documented so far. The current study 
investigates strategy use in a piece of self-developed IVD courseware (Liou, 
1994) by looking into two types of data based on comparison experiment design. 

Review of Literature 

A brief review of relevant literature is given first. The research in the 
first language (LI) reading strategies instruction reviewed by Grabe (1994) is 
probably among the most comprehensive in language learning fields so it was 
consulted to help us design our study. He found: 

1 . Activating prior knowledge does not automatically improve students’ 

comprehension of text or recall of information. Further research 
is needed. 

2. Question strategies which focus on central concepts, which show 

students how to find appropriate answers, which highlight text 
structure, which allow for inferential reflection and synthesis on 
a regular basis, which show students how to generate their own 
appropriate questions (and the purposes for doing this) seem to 
indicate student improvement on various outcome measures. 

3. Most research agrees that summary training, when done effectively, 

improves comprehension and recall, and shows transfer effects 
when students read other texts and on standardized reading 
comprehension measures (Flood & Lapp, 1990; Paris, et al., 

1991; Pearson & Fielding, 1991; Pressley, et al., 1989; Shih, 

1992; Taylor, 1992). 
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4. Monitoring is developmentally related; use of mental imagery, student 

verbalization, and underlining improve monitoring of text. 

5. Motivation and students’ perception of control over strategies appear 

to improve appropriate use of strategies and influence 
comprehension. Detailed explanation of when, where, and why 
to use strategies (conditional knowledge) is an important 
complement to what strategies to use and how to use them 
(procedural knowledge) (Duffy, 1993; Gaskin, et al., 1993; 
Pressley, et al., 1992; Garner, 1992). 

Grabe further raises some caveats for researchers: "much strategy training 
research uses very sensitive outcome measures— related to text; many cognitive 
training studies are not empirically based." Thus, the criterion measures where 
text-related items are used may be biased. In point 2, we realize that questioning 
strategies may help promote use of student metacognitive knowledge while 
facilitating the reading comprehension process. In point 3, the summary task, 
when used properly, may help comprehension as well. Our research design to 
be discussed later will try to respond to these caveats. 

Grabe also summaries the parameters which influence reading strategy 
research as follows: 

1. reader variables: developmental levels, skilledness, motivation for 

task, awareness of control over reading strategies, confidence, 
background knowledge (topical knowledge in reading), time 
spent, perception of benefits of training 

2. task variables 

material 

task (multiple-choice questions, or recall summary) 
past training, single/multiple tasks (what combination) 
difficulty level 

3. training variables 

focus strategies 
length of training 

means of assessing training outcome (post-training, 

interdependence of training) 

awareness of mental processes involved 

awareness of purpose for strategies 

intensity of training, sessions per week 

criterion level for successful training 

relation of training to other school tasks 
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amount of guidance in preparation for independent application 
amount of transfer effect to independent application 
opportunities for independent application 
amount of transfer effects to general comprehension assessment 
4. Content variables 

instruction time for strategy training 

relation of training to other curricular activities 

training of the teacher 

types of materials used in class 

teacher’s belief in efficacy of strategy training 

In implementing training of strategies, Winograd and Hare (1988) 
provide very useful guidelines for direct strategy training which has five 
components. 

1. What the strategy is. Teachers should describe critical, known 

features of the strategy or provide a definition/description of the 
strategy. 

2. Why the strategy should be learned. Teachers should tell students 

why they are learning about the strategy. Explaining the purpose 
of the lesson and its potential benefits seems to be a necessary 
step for moving from teacher control to student self-control of 
learning. 

3. How to use the strategy. Here, teachers break down the strategy, 

enact a task analysis for students, explaining each component of 
the strategy as clearly and as articulately as possible and showing 
the logical relationships among the various components. Where 
implicit processes are not known or are hard to explicate, or 
where explanatory supplements are desired, assists such as 
advance organizers, think-alouds, analogies, and ether attention 
clues are valuable and recommended (Roehler & Duffy, 1984). 

4. When and where the strategy is to be used. Teachers should 

delineate appropriate circumstances under which the strategy may 
be employed, (e.g., whether the strategy applies in a story or 
informational reading). Teachers may also describe inappropriate 
instances for using the strategy. 

5. How to evaluate use of the strategy. Teachers should show students 

how to evaluate their successful/unsuccessful use of the strategy, 
including suggestions for fix-up strategies to resolve remaining 
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problems, (pp. 123-124). 

Our design of strategies instruction to be discussed below follows this scheme. 
Similar guidelines and models are also recommended in Wenden (1991). 

Specifically for ESL or EFL training of strategies, there exist some 
training textbooks for students (e. g., Ellis & Sinclair, 1989; Rubin & 
Thompson, 1982; Willing, 1989), resource books for teachers (e.g., Cohen, 
1990; Oxford, 1990), learning training exercises incorporated into coursebooks, 
and supplementary materials (Sturtridge, 1989). 

For general strategies instruction, O’Malley et al. (1985; see also 
O’Malley, 1987), for instance, have trained ESL students three listening 
strategies: note-taking, selective attention for specific linguistic markers, and 
cooperation. The three represents a balance of three types of strategies: 
cognitive, metacognitive, and affective types. Their subjects included 75 ESL 
high schoolers. In the study, the metacognitive group received training of one 
metacognitive, two cognitive, and one socioaffective strategies; the cognitive 
group, only one cognitive and socioaffective strategies; and a control group, no 
strategy training. Subjects received the instruction and practice of strategies in 
a 50-minute session for 8 days. They found the interaction between strategy 
effectiveness and task difficultly: with more difficulty in the task, the training 
may fail to compensate for the complexity. They suggest that explicitness of 
directions for using the strategy is important for the effectiveness of training: 
"The transfer of strategies already used ... [requires] continued prompts and 
structured directions until the strategies become autonomous" (O’Malley et al., 
1985, p. 576). Wenden (1987) outlined a set of criteria that may guide learner 
training of strategies. 

CALL programs may promise a powerful environment, compared with 
human teachers’ instruction and student workbooks or worksheets, because they 
are extremely patient in providing continued prompts, powerful in incorporating 
structural directions, and adaptive in meeting individual student needs for the 
computer’s immediate feedback and robust speedy processing. As for research 
on training of learning strategies in the first language college students using 
CALL, Mikulecky, Clark, and Adams (1989) found that the reading strategy 
training designed in their computer program was effective, and the strategies 
could be transferred to new material. 

In EFL context, Chou (1992) investigated the effects of four kinds of 
help in a hypertext reading program: text only (no help), text with on-line 
vocabulary help, text with on-line sentence structural analysis, and text with 
strategy help (strategies of skimming, organization, guidelines to each paragraph. 



5 




6 



and supplementary information). He then randomly divided 39 high schoolers 
into four groups to work on the four versions. He found that types of help did 
not make a difference on reading comprehension among groups of subjects. 
However, his questionnaire data show that subjects think the vocabulary help 
was the most useful, and was used the most frequently. Meskill (1991) 
implemented strategy training in a multimedia CALL conversation database 
lesson, and documented her observation. Thirty-four English -as-a-second- 
language (ESL) college subjects were asked to use an IVD lesson with six types 
of advice messages implemented in the lesson; rehearse, monitor, repeat, plan, 
associate, and resource strategies. Another version of the IVD lesson did not 
incorporate strategy advice. Subjects were randomly assigned to either of the 
versions. On-line observation and off-line interview methods were used to 
collect the process data in order to answer the research questions. Results 
showed that most of the subjects read the advice messages and liked to follow 
the advice. As far as learning gain, time on task, and amount of material 
accessed, no significant differences were found between the advice and the non- 
advice groups. This study again confirms that learners vary with appropriate 
strategy use: good learners (with higher language proficiency) know what to do 
but weak learners need direct advice. 

II. THE STUDY 

The study proceeded at two stages. At the first stage, 81 subjects 
worked on the courseware individually. At the second stage, half of the 33 
subjects worked again individually on different content of the same design 
whereas another half worked in a small group. The entire study was conducted 
in a very much classroom-like situation although the design was experimental 
in nature. We feel that the design had exercised all the control we could have 
and the procedures were quite similar to what happens in other CALL contexts 
if language teachers would like to reinforce strategies instruction using 
supplementary CALL material. First, all the procedures were conducted in 
regular class time except when students needed to work on-line on a self-access 
basis. Namely, except the one-shot on-line session (about 50-70 minutes) in each 
stage, all the other activities were conducted in regular class sessions. Second, 
students were using different on-line versions based on their student 
identification numbers. The on-line session was designed in a way similar to that 
of the pre-test, posttest, and delayed posttest. In both stages, all subjects were 
asked to work on several multiple-choice or short-answer questions while on-line 
and to write a Chinese summary after they finished the viewing. Thus, students 
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would feel that they were working on different exercises at both stages. 

A. The First Stage: Effects of Strategies Instruction 

Eighty-one college freshman students from three majors— Chinese, 
Foreign Languages, and Economics— were recruited as subjects (we had had 
more but those who had viewed the video and those who lost data accidentally 
were not counted as valid subjects). Before the study started, we have collected 
the Chinese (the variable name referred to later as J-chi) and English (J-eng) 
scores of our subjects’ in their Joint College Entrance Examination taken in 
1994. We explained to the subjects the purpose and procedures of the study 
before asking them to sign up a consent form of participation. Then, after 
subjects viewed a 30-minute video clip twice, they took a pretest. It included 
a comprehension test with 20 multiple-choice questions in English and a Chinese 
summary recall task (between 1 10-130 Chinese characters in length). Meanwhile, 
a blank sheet of paper was given to each subject for the purpose of note-taking; 
subjects were encouraged to take notes if the act helped. After that, a 
questionnaire in Chinese (Questionnaire One) was given to survey all the 
subjects’ use of the strategies and beliefs of their use (See Appendix D). Then, 
a Chinese handout listing the names, explanations, and examples of the target 
strategies in the study was distributed. Subsequently, all the subjects were 
working individually on the on-line practice, to be discussed in detail shortly. 
The posttest followed exactly the same procedures except that subjects watched 
a different video clip and worked on different posttest items: multiple-choice 
questions (MC), recall summary, and Questionnaire Two. The total numbers 
of the MC items were the same. Three investigators in this research project also 
checked to make sure the items were appropriate and compatible. 

In the experiment, subjects were divided into two groups based on their 
student identification numbers. Then one group took the strategy version of the 
courseware (38 students in total), another (43 students) the non-strategy version. 
Three types of strategies were targeted in the treatment: metacognitive, 
cognitive, and affective strategies. Based on the results of our previous study, 
we selected "perceive visual/aural clues" and "use context" strategies. Based on 
our pedagogical experiences, we also selected the other common listening 
strategies which are deemed effective in viewing of feature films. In 
combination with the purpose of the viewing tasks-comprehension checks and 
summary recall, three strategies which have close connection with the paper-and- 
pencil tasks were taught at the same time: selective attention (read the questions 
in advanced before viewing), note-taking (on paper), and writing a summary (on 
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paper). The reason why we targeted at several strategies simultaneously was 
based on a belief that effective strategy use is derived from a combination of 
several and several types (cognitive, metacognitive, and socioaffective) together. 
Their categorization can be seen in Table 1. 

Table 1 Summary of Categorization of Strategies Trained 



metacognitive 


cognitive 


affective 


plan 


advance organizer 


*be confident 


monitor 


associate 




evaluate 


infer 




* selective attention 


predict 






perceive visual/aural clue 






segment -h recall 






use context 






* use imagery 






* take notes 






* summarize 





* those marked with * were not trained in the on-line practice. 



The metacognitive set was given on screen as a questioning strategy--see 
Table 2 for detail. Seven cognitive strategies were embedded with video content 
(the major strategy set). Three off-line strategies, the minor strategy set, are 
related to the overall viewing purpose: note-taking, summary recall, selective 
attention (read the off-line MC questions in advance prompted by computer text 
display). The cognitive strategy, "use imagery" and the socioaffective strategy, 
"be confident" were presented in the Chinese handouts. 



8 




9 



Table 2 Detail of Chinese Messages for the metacognitive s trategy group 



Planning 

1 . What do we need to do after viewing the film? 

2. What information is given to us? How can we 

make use of the information? 

3. Is there another way to do this? 

4. What should we do next? 

Monitoring 

1 , Are we using our plan or strategy? 

2, Do we need to change our plan, or use another strategy? 

3, Are we on the right track, close to our goal? 

Evaluating 

1. What worked? What didn’t work? 

2, How would we do differently next time? 

(King, 1991, p. 309) adapted from King’s "problem-solving prompt card" 



The strategies instruction in the study can be summarized as follows. 

1. A handout was given which explains the names of all 
the strategies and how they work before on-line 
practice. 

2. Subjects were exposed to on-line strategy messages 
including the list of the metacognitive strategy set, and 
contextualized cognitive strategy training. Because we 
had only three work stations, all the subjects took turns 
using the material only once for about one hour. 

3. Subjects were exposed to the off-line training of the 
three strategies combined with tosk requirements before, 
during, or after on-line practice: answering multiple- 
choice questions, taking notes, and writing a summary 
recall. The selective attention strategy was to ask 
subjects to read the multiple-choice questions before 
they viewed the video so that their attention would be 
focused on video information related to the questions. 

For the note-taking strategy, we gave blank sheets in the pretest. In the 
training section, the sheet for taking notes has some outline for the trainees in 
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the experiment group to follow as they wished. In the posttest tasks and the 
delayed posttest (in the second stage), we gave all the subjects the outline 
version. 

Our design followed the direct strategy training scheme (Winograd & 
Hare, 1988). All the subjects were given a handout with explanation and 
examples of all the target strategies. This serves the "what" component of the 
training. Very minimal instruction and illustration were given due to time 
constraint, but the importance was emphasized (the "why" component). The 
"when" and "v'here" components were implemented in the computer program. 
At appropriate points, for example, the program displays a message in Chinese 
such as "Can you predict what will happen next (in the video)?" or "What kind 
of person is so-and-so? If you think the character, so-and-so, is doing XXX, 
you are on the right track." The advance organizer strategy is to ask learners 
to view the video twice by getting a general impression the first time. The 
"how to use" and "how to evaluate strategy use" parts were not implemented 
in the project. A tracker program in the IVD unit was used to record how much 
time subjects spent in either round of video viewing. The strategy group used 
the version which was designed to require theni to view the video twice; the 
first viewing served as an opportunity for using the advance organizer strategy. 
The control group used the version in which subjects can decide for themselves 
if they , would view the video material once or twice. Among all the subjects, 
some even chose to view the material trice; all the decisions were made under 
student control. The total time for on-line viewing and strategy practice was 
referred to as TT. The time spent on first viewing Tl, that at the second time, 
T2. 



B. The Second Stage: Effects of Collaborative Strategies Learning 

In the second stage, only 33 out the original 81 subjects participated. 
They were all foreign language majors. Seventeen of the 33 still worked on the 
non-strategy version of a different video clip individually (i. e., the original 
control group), whereas sixteen of them worked in either pairs or triads together 
on the strategy version (i. e., the original experiment group). The 33 subjects 
took delayed posttest tasks, again answering multiple choice questions and 
writing a Chinese summary like what they did in pretest and p>osttest. Various 
on-line behaviors were recorded by the internal trackers designed in the IVD 
program. 

Except for the number of subjects and the design of small group on-line 
work, all the procedures in the second stage were the same as those in the first 
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stage. 



C. Data Analyses for the Two Stages 

Each of the multiple-choice questions (me) was assigned a different 
score from 1 to 3 points based on how much it demanded subjects to process 
the video. The assignment scheme worked like the following: 1 point for 
perceiving simple audio input, 2 for perceiving complicated audio input or 
inferring simple facts, 3 for inferring simple facts with difficult distractors in 
the questions or inferring complicated facts (see Appendix for a sample). The 
score assignment in the multiple-choice questions in the pretest, the posttest, and 
the delayed posttest made a total of 44 points for each of the tests. 

The summary protocols (S) were evaluated by a point system (Clark, 
1982; Irwin & Mitchell, 1983; Kalmbach, 1986). Three investigators together 
determined the crucial facts which should be included in each version of the 
summary tasks of the pretest, posttest, and delayed posttest sessions. The facts 
were further divided into more important idea units and important units. The 
important unit (I) was assigned 1 point, v.'hereas the more important unit (M), 
2 points. Three scorers read a sample of thirteen summaries randomly selected 
from all the data. Discrepancy was noted and discussed to achieve a consensus. 
Then, each of the summaries in all the data was read by two scorers 
respectively. When there was discrepancy again, the two scorers both met to 
resolve the difference. 

D. Results and Discussion 

A basic assumption for the comparison study is that the two groups of 
subjects which were compared should be drawn from a compatible population. 
Although the grouping was based on student identification numbers, the statistic 
cluster analysis using the multiple-choice test scores and summary scores in the 
pretest session may yield better grouping. (The cluster analysis groups together 
the observations with similar features in the sample into respective clusters.) 
Thus, the cluster analysis was conducted on the 81 subjects, and yielded three 
groups, high, mid, and low levels as in Table 3. The high level indicates the 
subjects in this cluster were the most proficient in English, and the low, the 
least proficient. 

The grouping was borne out by the "yes" responses in an item of 
Questionnaire One: "English is the subject whose grade was the highest in my 
high school". Then, both the T-test and the Wilcoxon rank sum test (w-test) 
were used to examine whether there were differences between the two groups 
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Table 3 Grouping Results of the Cluster Analysis (All the Subieclsl 



means 


no. of 
G1 


no. 

of 

G2 


J-chi 


J-eng 


mcl 


sl 


grade 


levels 


cluster 1 


28 


28 


75.2 


64.4 


28.8 


10.3 


34.5 

% 


mid 


cluster 2 


6 


3 


76.6 


65.6 


36.7 


10.7 


55.5 

% 


high 


cluster 3 


4 


12 


63.4 


61.2 


19.8 


9.9 


18.8 

% 


low 



no. of G1 and G2: number of the group 1, strategy group, and of the group 2, 
the non-strategy group in each cluster 

gradei!^: the frequency reported "yes" within each cluster for the item in 
Questionnaire One: "English is the subject whose grade was the highest in my 
high school". 

level^: the label given for each group after cluster analysis. 

The total scores of J-chi and J-eng are 100; MC: out of 44; SI: out of 21; S2: 
out of 20; S3: out of 18. 

in terms of J-chi, J-eng, me, s, m, i, tl, t2, and tt. Significant differences in 
the pretest session were found only in the variable of JCEE-Chinese scores for 
cluster 3 under the T-test. Since the group sizes are small (4 vs. 12), we feel 
confident to say that the two groups were the same before they received 
intervention or treatment. In the posttest session, significant differences were 
found only in the variables of tl for cluster 2, tl and tt for cluster 3 under the 
w-test. No differences were found in the summary, or multiple-choice 
performance. The significant statistics for the variables are listed in Table 4. 

Again, significance was found only in the small group size of cluster 
2 and cluster 3. A straightforward t-test was then conducted on all the original 
performance and the differential performance (difference between pretest and 
posttest) of the 81 subjects. None of the group differences were found in any 
of the dependent variables except tl and tt. In other words, as far the time on 
task in the on-line instruction is concerned, the strategy group spent more time 
completing the task than the non-strategy group, which was shown significant 
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Table 4 Comparisons of the Two Groups in Effects of Strategies Instruction 



cluster 


variables 


pretest/t-test 


posttest/w- 

test 


cluster 2 


T1 


NS(not 

significant) 


z =-2.204 

p =0.028* 


cluster 3 


J-chi 


t=2.709 

p=0.023* 


NS 


T1 


NS 


z=2.287 

p =0.022* 


TT 


NS 


z=2.02 

p=0.043* 



(p < .003; t=3.30; strategy group spent 49 minutes as a group but non-strategy 
group, 41). For the first-time viewing, we also found significant difference in 
the two groups (p <.05, t=2.28): the strategy group took 20.6 minutes as a 
group, the non-strategy group took 16.3 minutes. 

For subjects participating in the second stage, the cluster analysis was 
used again to classify the 33 students. It was found that only in the variables 
of the important idea units (I) in the summary recall and the total scores of the 
summary recall (S2) of only the posttest session were there significant 
differences between the two groups. The strategy group outperformed the control 
group in these two variables. Yet because the subject number was small, no 
conclusive answers could be achieved based on the statistical analyses. 

A straightforward t-test was then conducted on all the original 
performance and the differential performance (difference between pretest and 
posttest, between posttest and delayed posttest, and between pretest and delayed 
posttest) of the 33 subjects. None of the group differences were found in any 
of the dependent variables except the difference between the delayed posttest and 
the pretest of the multiple-choice items (MC3-MC1). To our surprise, the non- 
strategy group outperformed the strategy group (t=-2.9; p=0.007; the difference 
of the strategy group is 4.88, whereas that of the non-strategy group is 9.82). 
In the three-session assessment, there was an increase trend in the multiple- 
choice performance for the two groups as shown in Table 5. It was found that 
the strategy group, group 1, started with higher performance. Although the 
strategy group ended with a higher score, the performance difference between 
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the two groups was not significant. While the control group-the non-strategy 
group 2"Started with lower performance, they improved a great deal in the first 
stage. The phenomenon will be examined together with the results of the 
questionnaire data so that we can understand the effects of strategies instruction 
and collaborative learning in more depth. 

Table 5 Increases in Multiple-Choice Performance among Three Sessions For 
the Two Groups 



multiple- 

choice 


pretest 


1st stage 
improve- 
ment 


posttest 


2nd stage 
improve- 
ment 


delayed 

posttest 


group 1 


29.47 


2.7 


32.20 


3.1 


35.30 


group 2 


26.53 


4.5 


31.00 


2.8 


33.80 



total score = 44 



To sum up, statistical results confirmed the assumption that two groups 
were at the same baseline in the pretest given the condition that they were 
randomly divided into two groups. As for the effect of strategies instruction, 
while the strategy group did better in almost all the measures in the posttest 
sessions, multiple-choice items, summary writing, more important idea units, 
and important idea units (all the means are higher), the differences were not 
shown to be statistically significant. During the treatment period, the strategy 
group took more time on task due to either their own decision (using more on- 
line help) or the program control (requiring them to view the material twice), 
but the factor did not contribute to the improvement in the posttest in a 
significant way. In the collaborative learning stage, the non-strategy group made 
significant improvement during the pretest to the delayed posttest sessions, but 
the strategy group did not. 

Questionnaire data 

'We designed three questionnaires which were given at the pretest, 
posttest, and delayed posttest sessions respectively. The response in all the 
questionnaire items was elicited using a binary scale: yes or no. Answers of 
either category were cumulated for group comparisons. The chi-square test was 
used to examine if there was group difference. In about 30 items, we put two 
constructs, belief-of-strategy (BS) and habit-of-strategy-use (HSU) in the first 
questionnaire. In the second questionnaire, another two constructs, actual- 
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strategy-use (ASU) and like-to-use (LU) were incorporated. In Questionnaire 
Three, the same two constructs ASU and LU were used while incorporating 
some items on reactions towards collaborative learning (CL). 

Examples of belief-of-strategy (BS), habit-of-strategy-use (HSU), actual-strategy- 
use (ASU), like-to-use (LU), and collaborative learning (CL) 

BS: I believe that to understand English, inference making is important. 

HSU: To understand English, I often make inferences while listening. 

ASU: In this task, to understand English, I often make inferences 

while listening. 

LU: I like making inferences while listening to understand English. 

CL: I like to learn the computer language learning program with classmate(s). 

In Questionnaire One, there was no significant difference found between 
the two groups regarding the construct of belief-of-strategy (BS). However, the 
non-strategy group showed a lower piercentage of HSU compared with the 
strategy group: 59.0% vs. 71.4%. As for the habit-of-strategy-use (HSU), 
results indicate that in the only one item about a monitoring strategy the 
significant difference was found: the strategy group used more often (p =0.026). 
For the rest of the items about cognitive strategy use, there were group 
differences as well (p =0.000). The strategy group had higher ratios of using the 
strategies. The findings may suggest that the strategy group had believed in the 
usefulness of strategy use and had already been using the strategies during the 
pretest session; on the other hand, the non-strategy control group was probably 
not convinced by the usefulness of strategies, or did not use to, and did not use 
them as often as the strategy group at the initial stage. This may leave room for 
strategies instruction effects during the experiment period. This may explain why 
the non-strategy group improved in the delayed posttest multiple-choice measure 
(but not summary writing). Additionally, there was significant correlation found 
between BS and HSU (p=0.000). This may indicate that those believe in the 
usefulness of strategy use tended to use strategies in their learning. 

In Questionnaire Two, it was found that the on-line training set of 
strategies was used more than the off-line training set (82% vs. 52%). They 
seemed to like using metacognitive strategies more than cognitive ones in the 
future (85% vs.76%). The strategy group liked both cognitive and metacognitive 
strategy types more than the non-strategy group (85% vs. 75.5%). Those who 
used the strategies tended to like them for there was significant correlation found 
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(p =0.000). The correlation between ASU and LU was found to be significant 
(p= 0.005); this means those who really used strategies may indeed like to use 
strategies in the future in this sample. 

Table 6 Frequency Fluctuation of Note-Taking 



Groups 


pretest 


posttest 


delayed posttest 


group 1 


55.3% 


47.4% 


56.3% 


group 2 


44.2% 


53.5% 


41.2% 



The content of notes taken by all the subjects in the two stages during 
three sessions has not been analyzed in time for reporting. Yet, whether a 
subject took notes in each of the sessions was tallied. Their frequencies and the 
fluctuation were summarized in Table 6. To our surprise, fewer of the strategy 
group-group 1— took notes in the posttest session, i. e., after we provided on- 
line strategies instruction, while more of the control group took notes. It is 
likely that either the strategy group did not perceive note-taking as an effective 
strategy or they simply did not use the strategy on task regardless of its 
effectiveness. Whether the findings contributed to the causes for no-effect of our 
strategies instruction could not be answered given the note-taking data analyzed 
at its current stage. 

In questionnaire Three, none of the item or groups of items differed 
regarding responses in the two subject groups. Across the three questionnaires, 
significant differences were seen between responses of Questionnaire One and 
those of Questionnaire Two after we grouped all the items relating to strategy 
use (p=0.001), but none were seen between either those of Questionnaire One 
and Questionnaire Three, or those of Questionnaire Two and Questionnaire 
Three. 

Because we did not have enough IVD stations for the project, we had 
asked all subjects to work on the courseware only once for the treatment, or on- 
line practice of strategy training. Lack of enough practice is for sure the cause 
of no difference in the training effect. As for the effects of collaborative 
learning, against our expectation, the collaborative work did not prove better 
than individual work. The individual group, though not receiving strategy 
training, had exposure to the strategy handouts and learning material in two on- 
line sessions, which amounts to some form of input for their language 
acquisition. Moreover, the questionnaire responses indicate that the non-strategy 
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group did not use strategies before and may leave room for improvement due 
to IVD strategies instruction. Future studies should solve the resource limitation 
problems and let students use the computer material more. Because the increase 
trend is evident, given more practice, positive results in the training may be 
anticipated. It is also possible that the non-strategy group is more sensitized to 
strategy instruction than the actual strategy group. 

There are some implications we can draw from the study. In the study, 
two criterion tasks were used to evaluate students’ comprehension, instead of 
one; a combination of two tasks is helpful in obtaining a balanced measurement. 
In our training process, to focus strategies has been ensured in a Chinese 
handout. However, the length of training is short; namely, the intensity of 
training is weak. Thus, not only the amount of guidance in preparation for 
independent application but also instruction time is not enough. Further, 
students’ awareness of mental processes involved in using strategies was not 
raised though their awareness of usefulness of learner strategies was raised 
through both the on-line and off-line materials. In the future, the think-aloud 
demonstration and practice or an interview session can be conducted to for 
students to practice the use of effective strategies in appropriate situations. 

IV. CONCLUSION AND IMPLICATIONS 

In conclusion, this study confirms that the training of learning strategies 
is very sensitive to many variables. The greatest limitation in the study is the 
lack of enough training time, or learning time provided for learners. Yet, the 
study demonstrates a good example of incorporating CALL strategy training into 
regular curriculum, and the feasibilities of implementing strategy training in 
multimedia CALL courseware and using such material for individual learning 
as well as collaborative learning. 
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Appendix 

Posttest Questions 

*The assigned scores range from 1 to 3 points depending on their difficulty 

level. 

assigned score question item 

3 1 Ray responds, "I know what I crop." What does he really mean? 

a. I won't lose the farm. b. I crop my field of dreams, 

c. 1 crop corn. 

2 2. Why cannot the baseball players play a real game, according to Ray? 

a. They are ghosts. b. People cannot see them, 

c. There are only 8 of them. 

3 3. What is the second message from the corn field? 



2 4. How does the lady accuse Anne of making opposing comments? 

a. By saying "Your husband plowed the corn to build a baseball field." 

b. By commenting that she knew the answer beforehand. 

c. By making a bad response. 

3 5. What are all the arguments about? 

a. A writer's book b. Censorship 

c. Experiences in the 60s 
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